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(57) Abstract 

A fluorescent imaging system pro- 
vides an illuminated image of an object 
(14) that has been subjected to a fluores- 
cent dye, as well as a fluorescence image 
of the object Thus, highlighted fluores- 
cent portions of the object are viewed in 
context with the siurounding environment 
of the object being viewed. The imag- 
ing system (10) includes an optical system 
(20) which separates illumination light re- 
flected from the object from fluorescence 
light radiating from the object and pro- 
vides illuminated and fluorescence images 
of the object 
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FLUORESCENCE IMAGING SYSTEM 

Background of the Invention 

This invention relates to the detection of fluorescence and, in particular, the 
detection of fluorescence for cancer diagnosis. 

With conventional endoscopic techniques, early cancer detection presents a 
diagnostic challenge even for experienced physicians. In its early stages, a cancerous 
tumor may only be a few cell layers thick, and thus, the surface of the tissue may not 
physically change sufficiently to be visible when viewed under "white light" 
illumination with conventional endoscopic techniques. By "white light" illumination, is 
meant a relatively broad range of the visible portion of the electromagnetic spectrum. 

Fluorescence imaging techniques have been used to detect the presence of 
cancer. Fluorescence is generally defined as the emission of radiation, usually in the 
form of visible light, resulting from the absorption of radiation in another spectral band 
from a radiation source. The impetus for using fluorescence imaging in cancer 
detection was due in part to the recognition that normal and malignant tissue provide 
different fluorescence features. 

One approach for using fluorescence imaging in cancer detection involves 
comparing the intrinsic fluorescence (also referred to as "autofluorescence" or 
background fluorescence that is generated by all cells due to the natural presence of 
fluorescent molecules inside them) of normal tissue with the intrinsic fluorescence of 
malignantly-transformed tissue. 

Another approach for fluorescence imaging in cancer detection involves the use 
of exogenic substances, including fluorescent dyes, which are administered into a 
patient and accumulate in malignant tissue. Experiments with cultured cells indicate 
that the primary cause of dye accumulation in malignant tissue is due to the diflFerent 
metabolism rate of normal and cancerous cells. One advantage of this approach is that 
the fluorescence can be detected at a peak of the dye's fluorescence spectrum. 
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Summary of the Invention 

The invention features an imaging system which provides an illuminated image 
of an object that has been subjected to a fluorescent dye, as well as a fluorescence 
image of the object. Thus, highlighted fluorescent portions of the object are viewed in 
context with the surrounding environment of the object being viewed. For example, the 
invention can be used to detect the presence of cancerous tissue which are highlighted 
by fluorescence with the surrounding unfluoresced anatomy viewed m natural color. 

In a general aspect of the invention, the imaging system includes an optical 
system which separates illumination light reflected from the object from fluorescence 
light radiating from the object and provides illuminated and fluorescence images of the 
object. 

Embodiments of this aspect of the invention may include one or more of the 
following features. 

The imaging system includes a camera system which receives one or both of 
the illuminated and fluorescence images of the object. Images viewed without the 
camera system are viewed directly, for example, with an optical eyepiece. 

In one embodiment, the camera system is electronic (as compared with cam^as 
used for still photography) and generates electrical signals representative of the 
illuminated and fluorescence images. A first and a second image sensor are used to 
receive the illuminated and fluorescence images, respectively. 

The imaging system includes a display (e.g., CRT monitor) for viewing the 
illuminated and fluorescence images of the object. The display provides a 
superimposed view of the illuminated and fluorescence images of the object. 
Alternatively, the display is divided into regions, each of which is used to view a 
respective one of the illuminated and fluorescence images, for example, side by side. 

A processor receives and processes the electrical signals representative of at 
least one of the illuminated and fluorescence images of the object from the camera 
system and generates video signals for the display. 
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The optical system separates the fluorescmce light into a first wavelength 
range including the wavelength of maximum fluorescence of the fluorescent dye and a 
second wavelength range diflFerent from the first wavelength range. A first optical path 
receives the illumination light reflected from the object, a second optical path recdves 
5 the fluorescence light radiated from the object in the first wavelength range and a third 
optical path receives the fluorescence light from the object in the second wave length 
range. First and second optical filters are disposed in the second and third optical paths 
to limit transmission of fluorescence light in the first and second wavelength ranges, 
respectively. 

1 0 Because the fluorescence light fix)m the object is separated into two 

wavelength ranges the contribution of fluorescence associated with the 
autofluorescence can be minimized. The resulting fluorescent image, therefore, more 
accurately represents the fluorescence associated with the dye. Thus, a cancer- 
contrasting fluorescent dye which has accumulated in malignant tissue is highlighted 

1 5 with increased contrast without risk of being "masked" by the background 
autofluorescence. 

The second image sensor described above has a first region which recdves the 
fluorescence light in the second wavelength range and a second region which receives 
the fluorescence light in the third wavelength range. 

20 In one embodiment, the optical paths are provided by first and second 

beamsplitters. The first beamsplitter divides the reflected illumination light between the 
first image sensor and the second beamsplitter. The second beamsplitter divides 
illumination light received from the first beamsplitter to the first and second optical 
filters, respectively. 

25 Alternatively, in lieu of beamsplitters and individual optical filters, an electro- 

optical filter (e.g., a liquid crystal filter) is controlled to pass the reflected illumination 
light in a first time period, the first wavelength range in a first portion of the second 
time period, and the second wavelength range in a second portion of the second time 
period. 
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A spectral modulator is disposed between the light source and object to 
illuminate the object with white light and a non-white, spectral component of the white 
light (e.g., blue light) in an alternating manner. In one embodiment, the spectral 
modulator includes a rotatable filter wheel having a first region which passes the white 
5 light and a second region limiting transmission to the spectral component. 

The imaging system may be used to examine body tissue resected or otherwise 
removed firom the body. In this case, the imaging system includes a base for supporting 
the object under examination and a non-transparent enclosure attached to the base for 
enclosing the optical system and light source. An X-Y positioner is used to move at 
10 least one of the base and/or the optical system with respect to the other. An imaging 
system with this arrangement allows a surgeon in the operating room to examine the 
margins of the removed malignant/premalignant tissue to determine whether additional 
resection is required. The imaging system can also be used in conventional pathology 
laboratory settings. 

1 5 In another aspect of the invention, the imaging described above is utilized with 

a viewing scope for viewing the object under examination. "Viewing scope*' as used 
here is intended to mean any of the wide variety of instruments used to examine body 
tissue including endoscopes (e.g.,arthroscopes, horoscopes, laparoscopes) used to 
visualize interior portions of the body, as well as other external viewing devices, such 

20 as colposcopes. 

The vievwng scope includes the optical systrai described above, a light source 
to illuminate the object, a distal end enclosing a chamber and a lens system, disposed 
within the chamber, for viewing at least one of the illuminated image and the 
fluorescence image of the object. In an endoscope embodiment, the viewing scope 
2S includes an insertion section elongated along a longitudinal axis between the chamber 
at the distal end and a proximal end to be manipulated by a user. The optical system is 
coupled to the proximal end of the insertion section to receive the illuminated image 
and the fluorescence image from the lens system. 

In another aspect of the invention, a spectral modulator includes an input 
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dichroic beamsplitter which receives white light illumination from the light source and 
separates a selected color component (e.g., "blue" excitation illumination) from the 
white light illumination, an output dichroic beamsplitter which receives the separated 
selected color component from the first beamsplitter along a first light transmissive 
5 path and the remaining color components of the white light ilhimination along a second 
light transmissive path, and a shutter disposed in the second light transmission path and 
operable to selectively pass the selected color and the remaining color components 
from passing to the output dichroic beamsplitter. In operation, the shutter is controlled 
so that in a closed position, only the selected color component is provided by the 
10 spectral modulator. On the other hand, the shutter, inits open position, the remaining 
color components pass to the output dichroic beamsplitter where they are recombined 
with the selected color component to produce white light from the spectral modulator. 

Among other advantages, a spectral modulator of this construction does not 
have rotating optical parts, unlike those spectral modulators having filter wheels. The 
15 spectral modulator also provides near "lossless" spectral beam splitting. 

In another aspect of the invention, a spectral modulator includes a dichroic 
beamsplitter which is separated into two regions. A first region receives white light 
illumination from the light source and separates a selected color component from the 
white light illumination. The second region receives the selected color component 
20 along a first light transmissive path and the remaining color components of the white 
light illumination along a second light transmissive path, A shutter is disposed in the 
second light transmissive path and operable to selectively pass the remaining color 
components to the output dichroic beamsplitter. 

In addition to the advantages discussed above with respect to the spectral 
25 modulator having input and output dichroic beamsplitters, this construction requires 
only a single dichroic beamsplitter, thereby reducing the number of parts. 

In another aspect of the invention, a method of viewing an image of an object 
subjected to illumination light from a light source and a fluorescent dye, includes 
separating the illumination light reflected from the object from fluorescence light 
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radiating from the object and providing an illuminated image of the object and a 
fluorescence image of the object. Embodiments of this aspect of the invention may 
include one or more of the following features. 

The method includes displaying the illuminated image and the fluorescence 
5 image of the object on a monitor. For example, in one embodiment, the illuminated 
image and the fluorescence image are displayed as superimposed images of the object. 
Alternatively, the images are displayed on sub-divided regions of the display, one next 
to the other. Electrical signals representative of at least one of the illimiinated image 
and the fluorescence image of the object are generated. The electrical signals 
1 0 representative of the illuminated image and the fluorescence image of the object from 
the camera system are processed and video signals representative of the object for 
display are generated. 

Other features and advantages will become apparent from the following 
description and from the claims. 

15 Brief Description of the Drawings 

Fig. 1 is a block diagram of a fluorescence imaging system. 

Fig. 2 is a schematic representation, diagrammatic in part, of an endoscope 
embodiment of the fluorescence imaging system of Fig. 1 . 

Fig. 3 is a plan view of an image pickup device with laterally separated filter 
20 elements used in the embodiment of Fig. 2. 

Fig. 4 is a plan view of a rotating filter wheel used in the embodiment of Fig. 2. 

Fig. 5 is a graph depicting intensity of fluorescence as a fianction of wavelength 
for normal and malignant tissue administered with fluorescein. 

Fig. 6 is a graph depicting intensity of autofluorescence as a fixnction of 
25 wavelength for stomach tissue. 

Fig. 7 is a graph depicting a graphical approach for determining the wavelength 
selected for optimal detection of fluorescein in a background of autofluorescence. 

6 
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Fig. 8 is an another endoscope embodiment of the fluorescence imaging 

system. 

Fig. 9 is still another endoscope embodiment of the fluorescence imaging 

system. 

5 Fig. 1 0 is an embodiment of the fluorescence imaging system used in 

conjunction with a colposcope. 

Fig. 11 is an embodiment of the fluorescence imaging system enclosed within a 
chamber. 

Fig, 12 is an alternative embodiment of a spectral modulator. 
10 Fig. 13 is another embodiment of a spectral modulator. 

Description 

Referring to Fig. 1 , a fluorescence imaging system 10 (shown surrounded by a 
dashed line) is used in conjunction with an optical system 12 to view an object 14 that 
has been subjected to a fluorescent dye (e g ..fluorescein) and is being illuminated by a 

15 light source 16. As is described in greater detail below, light source 16 is modulated to 
alternately illuminate object 14 with "white" light providing normal visualization of the 
object and fluorescence excitation light (e.g., "blue" light) providing visualization of 
highlighted fluorescent portions of the object. The fluorescence excitation light is 
selected to be in a spectral range which provides a high level of florescence for the 

20 selected dye. 

In one embodiment, the fluorescence imaging system 10 superimposes a 
fluoresced image of object 14 obtained during illumination by blue light over a color 
image of object 14 during illummation by white light for view on a display 17. 

For example, in the field of cancer detection, object 14 is body tissue, such as 
25 an organ, being diagnosed for the presence of cancer tissue. The fluorescence imaging 
system 10 provides a surgeon with a fliU color image of a the body tissue contrasted 
with a superimposed fluoresced image of the body tissue indicating areas with cancer. 

7 
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As will be described in greater detail below, optical system 12 may be an endoscope 
for visualizing the organ within the body. 

Fluorescence imaging system 10 includes a spectral modulator 36 (described in 
greater detail below) and a beamsplitter assembly 1 8 having a first beamsplitter 20, 

5 such as a beam-splitting cube or mirror. First beamsplitter 20 passes a portion of the 
light from optical system 12 toward a color image pickup device 22 (e.g., a charge- 
coupled device (CCD)). Image pickup device 22 converts optical images of received 
light into electrical image signals and provides these signals to a color control unit 24. 
Color control unit 24 includes circuitry for controlling image pickup device 22 and 

10 image processing circuitry which converts the electrical image signals to digital signals 
for each pbcel of image pickup device 22. The digital signals from color control unit 24 
are received by a frame grabber 27 wWch captures the color image of object 14 on a 
frame-by-frame basis. A video tape recorder 25 may be connected to camera control 
unit 24 to provide a videotape recording of the examination, 

1 5 The remaining portion of the light not transmitted to image pickup device 22 is 

passed to a second beamsplitter 26 which provides a pair of laterally separated images 
onto the photosensitive front surface of a low-level light (LLL) image pickup device 
28. The increased sensitivity to light provided by this type of image pickup device is 
advantageous in detecting the fluoresced image which is generally of much lower 

20 intensity than the reflected light received by image pickup de>dce 22. The laterally 
separated images are passed through respective ones of a pair of bandpass filters 30, 
32 before being received by LLL image pickup device 28 which provides images of 
object 14 in a pair of preselected wavelength ranges. Bandpass filter 30 passes light at 
a peak wavelength associated with the maximum peak in the fluorescence spectrum of 

25 the particular fluorescent dye (e.g., fluorescein) injected or otherwise administered to 
object 14. Bandpass filter 32 passes light at an optimum wavelength for minimizing the 
contribution of light intensity associated with the autofluorescence of the object. 
Bandpass filters 30, 32 have a relatively narrow wavelength band of about 15mm 
wMch is less than approximately one third the spectral width of the fluorescence 

30 spectrum of the dye. 
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Analog electrical image signals representative of the filtered images are 
generated by image pickup device 22 and are provided to a LLL camera control unit 
34 where the image signals are combined and converted to digital signals. The digital 
signals from LLL control unit 34 are received by a frame grabber 35 which captures 
5 the fluoresced image of object 14 on a frame-by-frame basis. 

A SYNC circuit 37 provides synchronization signals to color control unit 24, 
LLL control unit 34, frame grabbers 27, 35 and spectral modulator 36 (e.g., a 
shuttering mechanism) which, as will be described in greater detail below, is used with 
light source 16 to alternately illuminate object 14 with white light and fluorescence 
10 excitation light (e.g., blue light). 

A processor 38 receives data from color camera control unit 24 and LLL 
control unit 34 via a bus 40 and manipulates the data to superimpose the color and 
fluoresced images received from camera control unit 24 and LLL control unit 34, 
respectively, for viewing on display 17. 

1 5 Referring to Fig. 2, in one embodiment of fluorescence imaging system 10, 

optical system 12 is in the form of an endoscope 50. In one application involving 
cancer detection, endoscope 50 is used to visualize an organ 5 1 having a malignant 
tumor. The organ has been injected or otherwise administered with a non-toxic, 
cancer-contrasting dye called fluorescein. As will be discussed in detail below, 

20 fluorescein is particularly well-suited as a cancer-contrasting dye due to its high 

fluorescence level, its non-toxidty, and its relatively high retention in cancerous tissue. 

Endoscope 50 has an elongated insertion section 52 (rigid or flexible) for 
insertion into a body cavity or narrow body passage to observe an organ 5 1 disposed 
therein. Endoscope 50 may be, for example, a Hysteroscope available from Smith & 
25 Nephew, Inc., Part No. 45 1 1 . Insertion section 52 extends along a longitudinal axis of 
endoscope 50 from a distal end 54 to a proximal end 56 where the image of organ 51 
is formed in the vicinity of a field stop 60 (or on the rear face of a coherent fiber 
bundle, in the case of a flexible endoscope). An ocular lens 62 positioned at proximal 
end 56 of endoscope 50, forms the virtual image of organ 51 at a normal viewing 



9 



wo 99/01-749 



PCT/US98/13567 



distance from proximal end 56. 

Beamsplitter assembly 58 includes first beamsplitter 20 (Fig. 1) in the form of a 
splitting cube 66 which reflects a portion of the light from endoscope 50 toward color 
image pickup device 22 (e.g., a color CCD) of a color camera head 68, such as Part 

5 No. 7204956, available from Smith & Nephew, Inc. Beamsplitter assembly 58 also 
includes a coupling lens 70 for focusing the reflected image onto the image plane of 
color image pickup device 22. As discussed above, image pickup device provides 
elearical signals representative of the color image of object 14 during illumination by 
white light. A cable 72 extends from camera head 68 for connection to color control 

10 unit 24, for example, Dyonics Digital Camera Control Unit, available from Smith & 
Nephew, Inc, Part No. 7204562. Color images observed at distal end 54 of endoscope 
50 are processed by the color control unit 24 and frame grabber 27 before being 
transmitted to processor 38 for viewing on display 17 (e.g., a color CRT monitor). 

The remaining portion of the light from endoscope 50 passes to LLL image 
1 5 pickup device 28 of a LLL camera head 74. The transmitted portion of this light is 
fiuther split into a pair of laterally spaced images by second beamsplitter 26 (Fig. 1) in 
the form of a pan- of optical wedges 76 laminated to the rear face of splitting cube 66. 
Optical wedges 76 are desired to be closely positioned with respect to the gkiX pupil of 
endoscope 50 to prevent vignetting (i.e., gradual fading at edge regions). Each of the 
20 laterally spaced images are passed through respective ones of filters 30, 32 to the 
photosensitive front surface of LLL image pickup device 28. 

Beamsplitter assembly 58 also includes a lens 78 for focusing the pair of 
laterally separated images onto the LLL image pickup device 28. Lenses 70 and 78 
may be variable focus or zoom type lens to accommodate endoscopes having field 
25 stops of various sizes. Beamsplitter assembly 58 has an outer housing 86 which, in 
combination with housings 88, 90 of color camera head 68 and LLL camera head 74, 
respectively, serve as a handle for allowing the user to position elongated insertion 
section 52 of endoscope 50 appropriately. 

Referring to Fig. 3, optical bandpass filters 30, 32, each selectively pass light 

10 
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from object 14 at relatively narrow ranges of wavelengths. The bandpass filters are 
produced, for example, using thin-film optical interference technology. In the case 
where fluorescein is used as the fluorescent dye, bandpass filter 30 passes light in a 
range between about 508 and 523 nra including a peak wavelength of 516 mm, the 
5 wavelength of the maximum peak in the fluorescence spectrum of fluorescein. 
Bandpass filter 32 passes light in a range between about 489 and 504 nm including 
wavelength 496 mm, the optimum wavelength for minimiang the contribution of 
fluorescence caused by autofluorescence, determined using a process described below. 

Referring again to Fig. 2, a cable 84 e5ctends from LLL camera head 74 for 
10 conveying signals representative of fluorescent images observed at distal end 54 of 
endoscope 50 to LLL camera control unit 34 and fi-ame grabber 35 where the signals 
are processed and then transmitted to processor 38 for viewing on display 17. 

A light guide cable 64, formed of a bundle of light guiding fiber optic elements, 
extends fi-om the side of proximal end 56 of endoscope 50 and conveys light, through 
1 5 spectral modulator 36 and from light source 16 to the area of organ 5 1 . In this 

embodiment, light source 16 may be a Xenon lamp, such as Smith & Nephew, Inc., 
Part No. 7205353. A suitable light guide cable 64 (Part No. 7205178), is also 
commercially available fi-om Smith & Nephew, Inc. 

Spectral modulator 36 includes a filter wheel 94 driven by a motor 96 and 
20 controlled by driver circuitry 98. Light from light source 16 is received by spectral 

modulator 36, via a relatively short fiber optic cable 92 and is transmitted through filter 
wheel 94 to light cable 64 through an optical coupler 100. 

Referring to Fig. 4, filter wheel 94 is formed of optical glass and is di\dded into 
two equal sized regions 102, 104 (separated along boundary lines 95a and 95b). 
25 Excitation filter region 102 passes "blue" excitation light in a spectral range between 
400-450 mm. Although visible blue light extends beyond this range, excitation filter 
region narrows the range of the blue excitation light to avoid possible interference with 
the fluorescent light emitted by the dye. Clear region 104, on the other hand, allows 
white light in a relatively broad wavelength range to pass. Although clear region 104 
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could be an open area, the use of clear optical glass is preferred so that filter wheel 94 
is mechanically balanced and compensation is not required. 

Filter wheel 94 rotates with the video frequency of display 1 7 (30 Hz for NTSC 
standard) and is synchronized with color control unit 24 and LLL camera control unit 

5 34 by SYNC circuit 37. In general, the fluorescent imaging system 10 is synchronized 
so that during one half of the video period required to scan a video fi-ame of display 
17, an image beam 106 incident on fiker wheel 94 passes through excitation filter 
region 102. During this half of the video period, fluorescent images from LLL image 
pickup device 28 are acquired by LLL camera control unit 34. During the other half of 

1 0 the video period, the image beam incident on filter wheel 94 passes through clear 

region 104 and LLL image pickup device 28 is turned off so that only color images are 
acquired by color control unit 24. In other words, LLL image pickup device 28 is 
electronically turned off when any part of image beam 106 is incident upon clear region 
1P4. Turning off LLL image pickup device 28 during this period protects it from high 

1 5 levels of illumination. On the other hand, color image pickup device 22 may be 
electronically turned oflF during "blue" illumination periods or may be left on 
continuously. In the latter case, the gain associated with the blue color channel for 
color camera control unit 24 is preferably reduced so that intensity level of blue is 
consistent with the red and green channels. 

20 The size of filter wheel 94 is dependent in large part on the size of the 

illuminating beam cross section. Thus, filter wheel 94 is closely positioned with respect 
to the light emitting face of fiber optic cable 92 so that the image beam Is relatively 
small. In certain applications, optical beam shaping elements may be used to focus light 
emanating from fiber optic cable onto filter wheel 94. Filter wheel 94 is also sized so 

25 that the video frame vertical blanking period (approximately 1.2 msec for NTSC 

standard) coincides with (or slightly exceeds) the time intervals during which boundary 
hnes 95a, 95b cross image beam 106 (as shown in Fig. 4). 

The following equation is used to determine the radius of the path on the 
surface of filter wheel 94 image beam 106 traverses: 

12 
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r = d/ ( (Trtvbfv ) + a 



(1) 



where 



d is the diameter of light beam 106; 



tvb is the vertical blanking period; 



5 



fv is the vertical scanning frequency (60 Hz forNTSC); and 



a is a correction factor for ensuring that the crossing of image beam 106 
with boundary lines 95a, 95b is within the vertical blanking period. 



Processor 38 receives and processes the color and fluoresced images received 
from color control unit 24 and LLL camera control unit 34, respectively, to provide a 

10 superimposed real-time image which is viewed on display 17. Each frame of the color 
image from color control unit 24 is digitized and stored by frame grabber 27. On the 
other hand, additional processing is required for processing the pzdr of laterally 
separated and filtered images which were passed through respective ones of bandpass 
. filters 30, 32. As discussed above, these filtered images represent the intensity of light 

15 at a wavelength of maximum fluorescence of the dye and the intensity of light at a 
wavelength selected for optimal discrimination of the autofluorescence from the object 
being viewed. Processor 38 processes the digitized fluorescent images on a pbcel-by- 
pixel basis using the following relationship: 



F = (Rcu-l2/Il)/(l2/Il-Rdyc) 



(2) 



20 



where 



F is the intensity of the pbcel; 



Ii is the fluorescence intensity at wavelength Xy of the maximum 
fluorescence of the selected dye; 



25 



I2 is the fluorescence intensity at wavelength for optimum 
autofluorescence discrimination; 
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Riis is the ratio I2/I1 for particular tissue being examined without dye; 

and 

Raye is the ratio I2/I1 for the particular dye. 

In general, equation (2) generates a calculated value F for each pixel which 
5 maximizes the contrast of fluorescence associated with the dye at wavelength ^1 by 
minimizing the contribution of background fluorescence emitted from the tissue at the 
optimum wavelength X2. Thus, the surgeon, can be confident that the superimposed 
fluoresced image being viewed represents fluorescence relating to the dye collected in 
the tissue, 

1 0 Bandpass filters 30, 32 ensure that the measurement of Ii and I2 are accurately 

performed at wavelengths X\ and X2, respectively. 

Rtis is specific for the particular type of tissue (e.g., stomach, bladder, skin) 
under examination. Both Rt« and Rdye are also specific to the measurement 
characteristics associated with the particular device (e.g., endoscope) used in the 
15 examination. Thus, a calibration procedure is typically performed with the device prior 
to establishing Rtjs and R^ye. Because the values ofXu Rti. and Rdye can be determined in 
advance, their values for specific dyes and types of tissue can be stored in look-up 
tables in a memory 39 (Fig. 1) associated with processor 38. 

It is important to note that equation (2) used by processor 38 to determine the 
20 florescence intensity is dependent solely on measurements at two wavelengths, X\ and 
X2. It is also an important advantage that the selection of wavelengths for X\ and X2 
fluorescence detection depend only on the particular dye and are independent of the 
type of tissue being examined. 

Wavelength X\ of the maximum fluorescence of the selected dye for bandpass 
25 filter 30 is determined simply fi^om a spectral analysis of the dye itself 

With reference to Figs. 5 and 6, an algebraic approach for obtaining the 
optimum discrimination wavelength X2 for bandpass filter 32 follows. 
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Referring to Fig. 5, the intensity spectra is shown for malignant tissue injected 
with fluorescein (line 110), normal tissue with fluorescein (line 1 12), normal tissue 
with fluorescein (line 112), and the autofluorescence of the tissue Oine 1 14). Note that 
the peak of maximum fluorescence for fluorescein at wavelength Xi{S\6 mm) is not 
5 coinddent with the peak of the spectrum for the malignant tissue. This is a result of the 
contribution of autofluorescence to the malignant tissue spectrum. 

The value of F calculated using equation (2) is actually representative of the 
ratio of intensity of fluorescence Ipi associated with fluorescein to the intensity of the 
autofluorescence lauio: 

10 F = If,/I.u,o (3) 

As shown in Fig. 5, In for malignant tissue is much greater than that of normal 
tissue. To show that equation (2) is equal to equation (3) the total fluorescence 
intensities for wavelengths X] and X2 are: 

IX, = I(,) = Il1 + Ia„to (4) 
15 1X2= 1(2) = Rdyclll + Rli, lauto. (5) 

The pair of terms in equation (4) correspond to the fluorescence intensities at X2 for 
the dye and tissue independently. Dividing equation (5) by equation (4) results in the 
following expression: 

I2/I1 = Rdyc (Ifl/Iauto) + RtU / afl/laulo) + 1 • 

20 Substituting F for Ifi/Iauio and solving for F provides equation (2). 

As shown in Fig. S, the value of F is different for normal and malignant tissues 
and is directly proportional to the concentration of dye present in the tissue under 
observation. 

The value of Rus cannot be determined during fluorescence imaging or in 
25 advance because the value for different types of tissue as well as samples of each 

particular type of tissue vary. However, a value of Rtj, for the particular type of tissue 
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can be determined using statistical techniques. In particular, the average value and the 
scatter (i.e., the statistical variation) of Rus for the same type of tissue may be 
measured. The dependencies of both of these parameters on wavelength are 
determined by statistical processing of autofluorescence spectra, normalized to the 
intensity at wavelength Xi, 

Referring to Fig. 6, an average normalized spectrum, t (X) = Ru. (X) of the 
autofluorescence (line 1 16) is provided based on measurements of various samples of 
stomach tissue shown. The average spectrum plus and minus one standard deviation of 
the measurement of autofluorescence across the wavelength of interest (i.e., t(X) + s(X) 
and t{X) - s(k) are represented by line 1 18 and line 120, respectively. Lines 118 and 
120 represent the scatter of Rti, (K). Line 122 represents the normalized intensity 
spectra for fluorescein solution fl[X) that is the same as R<iyc (A*). 

Using the standard deviation s of Riia, the value of F is determined from 
equation (2) with the standard error: 

(1+F) (s/(Rtis-R<lyc)) 

Thus, the optimum wavelength X2 for detection of dye can be determined by 
minimizing an error factor % (X) which is expressed as follows: 

X(X) = s(X)/(Rti»NX)-Raye(X)) = x(X)/(t(X)-f(X)) (6) 

X'{^) = 0 (7) 

As shown in Fig. 6, functions s(A.) and t{X) are suflBciently smooth (i.e., have no sharp 
discontinuities) to allow s(X) and t(X) to expressed about point X\ as: 

s(X)=s'(}i,) (X-Xi); 

t(X)=l+t^ (X,)(X-?ii) 

where &\X{) and f(X.i) are the derivatives of s(X) and t(X). Substituting these 
expressions into equation (7), the minimum value for % (X) is determined to be at a 
wavelength X2 for which the following equation is true: 

16 
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1 -fl[>.2) = f(X2)(>Ll-X2). (8) 

From equation (8) it can be seen that the optimal discrimination wavelength depends 
only on the dye spectrum f (k) and is independent of s' (K\) and t'(Xi) which may vary 
for different dye-containing tissue. 

S Referring to Fig, 7, a graphical representation of the solution to equation (8) 

provides an approach for determining wavelength ^2. The graphical solution can be 
performed geometrically by starting at the wavelength of maximum fluorescence Xi for 
fluorescein dye, which is at the peak (point 126) of spectrum curve 122. A line 124 is 
then drawn tangent to the spectrum curve and through point 126. The point of 

10 tangency provides the optimum value for X2 which for fluorescein is 496 nm. 

Other embodiments are within the claims. For example, referring to Fig. 8, a 
handle 130 for manipulating endoscope 50 is of reduced size. This reduction in size is 
accompUshed by separating first beamsplitter assembly 20 from second beamsplitter 
26. In this embodiment, the beam-splitting arrangement is different from that described 

15 above in conjunction with Fig. 2. In particular, splitting cube 66 remains in a housing 
1 32 while second beamspHtter 26, enclosed within a housing 141, includes a single lens 
142 for projecting the image from the proximal face of fiber bundle 140 onto a pair of 
rhomboidal prisms 144, 146 having laminated on their rear faces, bandpass filters 30, 
32, respectively. Second beamsplitter 26 is optically coupled to beamsplitter assembly 

20 20 with a coherent fiber bundle 140. With this arrangement, LLL camera control unit 
34 and second beamsplitter assembly 26 can be remotely located from endoscope 50, 
for example, on a surgical cart. 

Referring to Fig. 9, a temporal approach for separatmg the color and fluoresced 
images is used rather than the optical beam-splitting approaches described above in 
25 conjunction with Figs. 1 and 8. In this embodiment, atunable filter assembly 150 having 
a lens relay system 152 and tunable filter 154 is connected to the proxmud end 56 of 
endoscope 50. The light transmitted through tunable fiUer 154 is received by a LLL 
image pickup 156 of a LLL camera head 158. Tunable filter 154 is controlled by a 
filter driver 159 to divide each image frame into the required full color and 
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fluorescence image acquisition periods. During the full color acquisition period, 
tunable filter 154 is controlled to pass white light (including red, green and blue (RGB) 
components images to provide the full color image. During the fluorescence image 
acquisition period, tunable filter 154 is controlled to pass images at (1 and (2 within 
5 the blue portion of spectrum. 

The system of this embodiment is synchronized so that LLL camera head 158 is 
switched by processor 38 (Fig. 1) to a low gain mode during the full color acquisition 
period and to high gain mode during the fluorescence image acquisition period. 
Alternatively, the relative time periods for full color and fluorescence image acquisition 

10 can be varied. The acquisition time periods may be varied on the basis of the intensity 
of light available from light source 16 and/or the light eflBciency of endoscope 50. For 
example, when light intensity is high, only one third of the video frame time may be 
allocated for full color (RGB) acquisition and correspondingly, two thirds for 
fluorescence. For low light intensities this relationship could be reversed. The timing 

1 5 allocations can be set automatically by processor 38 during calibration or adjusted by 
the user. 

Image acquisition periods for specific wavelength ranges within the full color 
and fluorescence image periods can also be varied. For example, the time periods for 
obtaining red, green and blue light can be varied to correspond to the levels of human 
20 eye sensitivity to the red, green and blue spectral bands. Generally, the time period for 
obtaining green light would be longest and shortest for blue light. 

A wide variety of image sensors are applicable for use in LLL camera head 158 
including Intensified CCD (ICCD), back-illuminated CCD (BCCD), electron- 
bombarded back-illuminated CCD (EBCCD). However, in the embodiment of Fig. 9, 
25 BCCD or EBCCD image sensors are particularly advantageous, since the same image 
sensor is used to acquire both the full color and fluorescence images. A BCCD image 
sensor is advantageous because high voltages are not required, but is not 
recommended where a low power light source is used or the endoscope is too small to 
yield enough radiation energy. 
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Tunable filter 1 54 may be electromechanical (e.g., a filter wheel) or be based 
on electro-optical principles (e.g., a liquid crystal). Tunable liquid crystal filters are 
advantageous because they do not require moving parts and are of small size. 
Ferroelectric liquid costal tunable filters are particularly advantageous in applications 
5 requiring fest switching speeds (in the order of microseconds) allowing the fluorescent 
imaging system to perform in real time. 

Referring to Fig. 10, fluorescent imaging system 10 is used with a colposcope 
160 for gynecological examination including examining cervical or vaginal tissue 161. 
Tissue 161 is illuminated by an illuminator 162 which receives light via a fiber optic 

10 cable 163 firom light source 16 and spectral modulator 36. Spectral modulator 36 
provides blue and Ml color (white light) in the alternating manner described above. 
Reflected light firom tissue 1 6 1 received by a first beamsplitter 1 64 is divided into two 
paths, a first portion passed directly to the binocular viewing device of colposcope 160 
with the remaining light reflected to a second beamsplitter 165. A yellow filter may be 

15 placed in the optical path between first beamsplitter 164 and the viewer to offset overly 
blue light caused by spectral modulation of the light source. Beamsplitter 164 may also 
be included within colposcope 160. The light received by second beamsplitter 165 is 
divided between a color camera head 166 and a third beamsplitter 167 similar to the 
LLL beam splitting device described in conjunction with the embodiment of Fig. 8. 

20 Third beamsplitter 167 includes bandpass filters 168a, 168b which pass light at 
wavelengths associated with the fluorescent dye and optimum autofluorescence, 
respectively. The filtered images are received by a LLL camera head 169 and, along 
with the fiiU color image fi^om color camera head 166, are further processed by 
processor 38 in the manner described above. 

25 Any of the fluorescent imaging systems described above in conjunction with 

Figs. 2, 8 and 9 may be adapted for use in analyzing tissue removed fi-om the body. For 
example, referring to Fig. 11, a system 170 includes a tray or substrate 172 supported 
on a base 174. The object, such as resected tissue 176 (shown in phantom), is placed 
on the substrate 172 and covered by a protective glass plate 178. The components of 

30 fluorescent imaging system 10 (shown enclosed within dashed lines of Fig. 1) are 
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mounted to an X-Y positioner 1 80 above substrate 172 and resected tissue 176. 
Alternatively, fluorescent imaging system 10 may be stationary with substrate 172 
mounted to the X-Y positioner. 

An optical assembly 182 provides a focused image of the resected tissue to 
fluorescent imaging system 10. System 170 is enclosed within a non-transparent 
housing 184 which with base 174 provides an examination chamber to prevent 
illumination of the resected tissue with ambient room light. Light source 16 and 
spectral modulator 36 are positioned within housing 10. A remote control box 188 and 
processor 38 (Fig. 1) are connected by cables 186, 188 to fluorescent imaging system 
10, X-Y positioner 180, optical assembly 182, light source 16 and spectral modulator 
and provide appropriate control, power and synchronization signals there between. 

System 170 is particularly well-suited for examining the margins of resected 
malignant/premalignant tissue to determine whether additional resection is required. 
System 170 may be used in the operating room as well as in conventional pathology 
laboratory settings. Because system 170 is not manipulated within the body, the small 
size requirements and real-time performance are not always necessary. Indeed, in 
certain applications, the video cameras described above may be replaced with still 
cameras. 

Referring to Fig. 12, an alternative embodiment of a spectral modulator 190 
includes a first dichroic beamsplitter 192 which receives and divides incoming white 
light into two components. The first component is a blue component in the wavelength 
range between about 400-450 nm to a first mirror 194 where it is reflected toward a 
second dichroic beamsplhter 196. The second component includes red and green 
components which are reflected by transmitted towards a second mirror 198 where it is 
reflected by first dichroic beamsplitter 192 toward second dichroic beamsplitter 194. 
Second dichroic beamsplitter 196 has a construction which is different than that of first 
dichroic beamsplitter 192 and functions to reflect the blue component and pass red and 
green components. An "on-ofF' shutter 200, positioned between second dichroic 
beamsplitter 194 and second mirror 198, is controlled and synchronized by processor 
38 via a shutter driver 202. The construction of the shutter 200 may be based on 
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electromechaJiicaJ or electro-optical principles. Shutter driver 202 is synchronized so 
as to keep the shutter 200 open during full color image acquisition and closed during 
fluorescence image acquisition. 

In operation, when shutter 200 is closed, spectral modulator 190 transmits only 
5 blue light for fluorescence excitation. On the other hand, when shutter 200 is open, the 
green and red light reflected from second mirror 198 is recombined with the blue light 
from first mirror 194 at first dichroic beamsplitter 192 to reproduce the white light. 
Spectral modulator 190 provides near "lossless" spectral beam splitting and an 
arrangement without rotating optical elements. 

10 Referring to Fig. 13, a spectral modulator 210 requiring even fewer optical 

elements than spectral modulator 190 of Fig. 12 is shown. Spectral modulator 210 
includes a dichroic beamsplitter 212 having a first portion 214 which receives white 
light from light source 16 and reflects blue light toward a first mirror 216 where it is 
reflected toward a second portion 218 of dichroic beamsplitter 212. The remaining red 

15 and green light incident on first portion 214 passes through dichroic beamsplitter 212 
where it is reflected by a second mirror 220 toward second portion 218. 

A shutter 222 is positioned between second portion 218 and second mirror 220 
and is controlled and synchronized by processor 38 (Fig. 1) to keep shutter 222 open 
during full color image acquisition and closed during fluorescence image acquisition. 

20 Thus, as was the case in the embodiment of Fig. 12, when shutter 222 is closed, 

spectral modulator 210 transmits only blue light for fluorescence excitation and when 
shutter 222 is open, the green and red light reflected fi-om second mirror 220 is 
recombined with the blue light from first mirror 216 at second portion 218 to produce 
the white light. Spectral modulator 210 advantageously provides better quenching of 

25 residual green and red components in the blue light than that provided by spectral 
modulator 190 because the blue light is reflected twice by dichroic beamsplitter 212. 

The use of fluorescein as a cancer-contrasting dye was used in the descriptions 
of the above embodiments. However, the imaging systems described above can use 
other dyes including hematoporphyrin and its derivatives. The approach described 
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above for improving the contrast of a fluorescent image by minimizing the contribution 
of fluorescence due to autofluorescence is equally applicable to fluorescent dyes oth^ 
than fluorescein. 
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Still other embodiments are within the scope of the claims. What is claimed is: 

1 . An imaging system for viewing an image of an object under 
examination, the object subjected to illumination light from a light source and a 
fluorescent dye, the imaging system comprising an optical system which separates the 
illumination light reflected from the object from fluorescence light radiating from the 
object and provides an illuminated image of the object and a fluorescence image of the 
object. 

2. The imaging system of claim 1 further comprising a camera system 
which receives at least one of the illuminated image and the fluorescence image of the 
object. 

3. The imaging system of claim 2 wherein the camera system includes an 
electronic camera which generates electrical signals representative of the at least one of 
the illuminated image and the fluorescence image of the object. 

4. The imaging system of claim 3 wherein the camera system includes a 
first image sensor which receives the illuminated image and a second image sensor 
which receives the fluorescence image. 

5. The imaging system of claim 1 fiirther comprising an optical eyepiece 
for viewing at least one of the illuminated image and the fluorescence image of the 
object. 

6. The imaging system of claim 2 further comprising a display for viewing 
the illuminated image and the fluorescence image of the object. 

7. The imaging system of claim 6 wherein the display provides a 
superimposed view of the illuminated image and the fluorescence image of the object. 

8. The imaging system of claim 6 wherein the display is divided into a first 
and second display regions, the first region for viewing the illuminated image and the 
second region for viewing the fluorescence image of the object. 
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9. The imaging system of claim 6 fiirther comprising a processor which 
receives and processes the electrical signals representative of the at least one of the 
illuminated image and the fluorescence image of the object from the camera system and 
generates video signals for the display. 

10. The imaging system of cl^m 1 wherein the optical system separates the 
fluorescence light separated into a first wavelength range including the wavelength of 
maximum fluorescence of the fluorescent dye and a second wavelength range different 
from the first wavelength range. 

1 1 . The imaging system of claim 10 wherein the optical system includes: a 
first optical path for receiving the illumination light reflected from the object; a second 
optical path for receiving the fluorescence light from the object in the first wavelength 
range; and a third optical path for receiving the fluorescence light from the object in 
the second wavelength range. 

12. The imaging system of claim 1 1 fiarther comprising: 

a first optical filter disposed in the second optical path to limit transmission of 
fluorescence light in the first wavelength range; 

and a second optical filter disposed in the third optical path to limit 
transmission of fluorescence light in the second wavelength range. 

1 3 . The imaging system of daim 1 0 wherein the camera system includes a 
first image sensor which receives the illuminated image and a second image sensor 
which receives the fluorescence image. 

14. The imaging system of claim 13 wherdn the second image sensor has a 
first region which receives the fluorescence image in the second wavelength range and 
a second region which receives the fluorescence image in the third wavelength range. 

15. The imaging system of claim 14 fiirther comprising a first beamsplitter 
and a second beamsplitter, wherein the first beamsplitter divides the reflected 
illumination light between the first image sensor and the second beamsplitter; the 
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second beamsplitter divides illumination light received from the first beamsplitter to the 
first and second optical filters, respectively. 

16. The imaging system of claim 1 further comprising an electro-optical 
filter controlled to pass the reflected illumination light in a first time period, the first 
wavelength range in a second time period, and the second wavelength range in a third 
time period. 

1 7. The imaging system of claim 1 6 wherein the electro-optical filter is a 
tunable liquid crystal filter. 

18. The imaging system of claim I fiirtiier comprising a spectral modulator, 
disposed between the light source and object, to altematingly illuminate the object with 
white light and a spectral component of white light. 

19. The imaging system of claim 1 8 wherein the spectral modulator 
includes a rotatable filter wheel having a first region which passes the white light and a 
second region limiting transmission to the spectral component of white light. 

20. The imaging system of claim 1 fijrther comprising a base for supporting 
the object under examination and a non-transparent enclosure attached to the base for 
enclosing the optical system and light source. 

21 . The ima^g system of claim 20 further comprising an X-Y positioner 
for moving at least one of the base and optical system with respect to the other. 

22. A viewing scope for viewing an image of an object under examination, 
the object subjected to a fluorescent dye, the viewing scope comprising: a light source 
to illuminate the object; a distal end enclosing a chamber, an optical system which 
separates the illumination light reflected from the object from fluorescence light 
radiating from the object and provides an illuminated image of the object and a 
fluorescence image of the object; a lens system, disposed within the chamber, for 
viewing at least one of the illuminated image and the fluorescence image of the object. 

23. The viewing scope of claim 22 fiirther comprising: an insertion section 
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elongated along a longitudinal axis between the chamber at the distal end and a 
proximal end to be manipulated by a user, the optical system coupled to the proximal 
end of the insertion section to receive the illuminated image and the fluorescence image 
from the lens system. 

24. A spectral modulator comprising: an input dichroic beamsplitter which 
receives white light aiimiination from the light source and separates a selected color 
component from the white light illumination; an output dichroic beamsplitter which 
receives the selected color along a first light transmissive path and the remaining color 
components of the white light illumination along a second light transmissive path; a 
shutter disposed in the second light transmissive path and operable to selectively pass 
the selected color and the remaining color components from peissing to the output 
dichroic beamsplitter. 

25. A spectral modulator comprising: a dichroic beamsplitter having: a 
first region which receives white light illumination from the light source and separates 
a selected color component from the white light illumination; and a second region 
which receives the selected color component along a first light transmissive path and 
the remaining color components of the white light illumination along a second light 
transmissive path; and a shutter disposed in the second light transmissive path and 
operable to selectively pass the remaining color components to the output dichroic 
beamsplitter. 

26. A method of viewing an image of an object under examination, the 
object subjected to illumination light from a light source and a fluorescent dye, the 
method comprising: separating the illumination light reflected from the object from 
fluorescence light radiating from the object; and providing an illuminated image of the 
object and a fluorescence image of the object. 

27. The method of claim 26 further comprising displaying the illuminated 
image and the fluorescence image of the object on a monitor. 

28. The method of claim 26 wherein the displaying step includes 
superimposing the illuminated image with the fluorescence image of the object. 
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29. The method of claim 27 further csomprising generating electrical signals 
representative of the at least one of the illuminated image and the fluorescence image 
of the object. 

30. The method of claim 29 further comprising: 

processing the electrical signals representative of the illuminated image and the 
fluorescence image of the object from the camera system; 

and generating video signals representative of the object for display. 
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This International SearchiDg Authority found multiple inventions in this international application, as follows: 
Please See Extra Sheet 



1 . I I As all required additional search fees were timely paid by the applicant, this international search report covers all searchable 

claims. 

2. I I As all searchable claims could be searched without effort justifying an additional fee, this Authority did not invite payment 

of any additional fee. 

3. I I As only some of the required additional search fees were timely paid by the applicant, this international search report covers 

only those claims for which fees were paid, specifically claims Nos.: 



4. I x| required additional search fees were timely paid by the applicant Consequently, this international search report is 
restricted to the invention first mentioned in the claims; it is covered by claims Nos.: 
1-23. 26-30 



Remark on Protest | | The additional search fees were accompanied by the applicant's protest 

I I No protest accompanied the payment of additional search fees. 
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INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US98/13567 



BOX n, OBSERVATIONS WHERE UNITY OF INVENTION WAS LACKING 
This ISA found multiple inventions as follows: 

This application contains the following inventions or groups of inventions which are not so linked as to form a single 
inventive concept under PCT Rule 13.1. In order for all inventions to be searched, the appropriate additional search fees 
must be paid. 

Group I, claim(s) 1-23 and 26-30» drawn to systems and method to view images of an object subjected to fluorescent 
dye, classilied according to the national classification in Class 250, Subclass 458.1. 

Group II, claim(s) 24 and 25, drawn to a spectral modulator, classified according to the national classification in Class 
359, Subclass 634. 



The inventions listed as Groups I and 11 do not relate to a single inventive concept under PCT Rule 13.1 because, under 
PCT Rule 13.2, they lack the same or corresponding special technical features for the following reasons; the separation 
of reflected illumination light from fhiorescence emission is not found in the invention of Group H. 



Fonn PCT/ISA/210 (extra sheetXJuly 1992)* 



